\chapter{Test setups and results from prototype benchmarking}
\label{appendix}
\section{Built-in functions}
Most built-in functions showed few performance gains in our earlier tests. Matrix multiplication was an exception and showed a potential performance gain, so we have tested it in our prototype.

\begin{figure}[H]
\includegraphics[width=1.0\textwidth]{pics/CoreCalcMMUlt.png}
\caption{Matrix multiplication test setup}
\label{mmult in corecalc}
\end{figure}

As can be seen from the figure, matrix multiplication was implemented to take two equally sized quadratic arrays of random numbers. The CPU is initially faster with an array size of $8^2$ (64). The GPU starts getting faster at size $96^2$ (9216). For the sample we have made, the time spent by on the GPU seems to grow linearly as a function of data sizes, while the CPU has a higher complexity in its growth.

\begin{figure}[H]
\includegraphics[width=1.0\textwidth]{pics/mmult.pdf}
\caption{Matrix multiplication performance results}
\label{mmult in corecalc results}
\end{figure}

The GPU is five times faster that the CPU at a size of $240^2$ (57600), which is the largest possible dataset the GPU can handle during matrix multiplication. As mentioned we have no indicator of when this maximum is reached, but our test results.


\section{Sheet defined functions}
\subsection*{Herons formula}
Herons formula has been implemented as a sheet defined function and tested on data sizes from 1000 to 50000 floats.

This implementation has been used a solid base for further tests, to see how different changes affect the performance.

\begin{figure}[H]
\includegraphics[width=1.0\textwidth]{pics/HeronParamRandom.png}
\caption{Different variations of Herons formula in CoreCalc}
\label{herons in corecalc}
\end{figure}

\subsubsection{Random data}
In this implementation all three variables (A,B,C) in Herons formula are random data generated by \keyword{rand()} for each invocation. This means three unique FPAs are generated and sent to the GPU. Tests are run on data sizes from 1000 to 50000 floats, and an intersection was not found. Using linear regression we can derive the following functions:

$HeronRandomCPU(x) = 452,4939*x-850988$
$HeronRandomGPU(x) = 393,2681*x+2847012$


It can be seen, that though our tests do not show an intersection, because of the slopes, it will be hit well within the limits of the possible data on the GPU.

\begin{figure}[H]
\includegraphics[width=1.0\textwidth]{pics/HeronRandom.pdf}
\caption{Herons Formula with random data, performance results}
\label{mmult in corecalc results}
\end{figure}

Notice that with random values for A, B and C, not all generated examples will be able to create a triangle. We might end up with a negative value in the SQRT, which might impact speeds on both the CPU and the GPU, making the results harder to actually compare to the two other Heron implementations. 

\subsubsection{Param data}
\begin{figure}[H]
\includegraphics[width=1.0\textwidth]{pics/HeronParam.pdf}
\caption{Herons Formula with parameter data, performance results}
\label{mmult in corecalc results}
\end{figure}
In this implementation A and B are set as parameters and C is set to $A+B/2$. This way only two FPAs are sent to the GPU, along with an operation and a single constant number. An intersection was not found. Using linear regression we can derive the following functions:

$HeronParamCPU(x) = 462,4716*x-971310$
$HeronParamGPU(x) = 457,2948*x+4481264$

It can be seen, that though our tests do not show an intersection, because of the slopes, it will be hit well within the limits of the possible data on the GPU.

\subsubsection{Constant data}
\begin{figure}[H]
\includegraphics[width=1.0\textwidth]{pics/HeronConstant.pdf}
\caption{Herons Formula with constants data, performance results}
\label{mmult in corecalc results}
\end{figure}
In this implementation all values are the same constant. Only one constant is transferred to the GPU. . An intersection was not found. Using linear regression we can derive the following functions:

$HeronConstantCPU(x) = 456,901*x-933632$
$HeronConstantGPU(x) = 440,3515*x+3378552$

It can be seen, that though our tests do not show an intersection, because of the slopes, it will be hit well within the limits of the possible data on the GPU.
\subsection{Fibonacci sequence}
A given number in the Fibonacci sequence can be calculated using this formula.

$F_n = \frac{1}{\sqrt{5}} \left(  \left( \frac{1 + \sqrt{5} }{2}\right)^n -\left(\frac{1 - \sqrt{5}}{2}\right)^n \right)$

\begin{figure}[H]
\includegraphics[width=1.0\textwidth]{pics/fibonacci.pdf}
\caption{Performance test of the Fibonacci sequence}
\end{figure}

We implemented this as a simple sheet defined function in CoreCalc. The SDF uses 4 constants and one input value. The test is run from 1000 floats to 30.000 floats. The GPU surpasses the CPU around a size of 15.000.

\subsubsection{Monte Carlo simulations}
Monte Carlo simulations are used in a wide variety of industries to estimate probable outputs where deterministic algorithms would take too long to compute or simply be too complex. They rely on a random sample data and statistics, and are often implemented in Microsoft Excel. 

While we have implemented the data-sampling in the simulations below, we have not spent time on calculating the actual results, because of the relatively few aggregate functions implemented in CoreCalc. 

Aggregate functions will be reduction operations on the GPU which are often not very effecient. The lack of actual results might skew the results of an actual simulation, however we believe these results give a good guidance since the aggregation will often be faster to do on the CPU and a users might want to analyse the result in many ways.

\subsubsubsection{Greeting card estimation}
Straight out of an example from Microsoft (http://office.microsoft.com/en-us/excel/HA011118931033.aspx) of how Monte Carlo can be used to make business decisions, simulating different types of demand scenarios, outputting what the risk is of failing.

We converted this example into a SDF and did the same simulation in CoreCalc.

\begin{figure}[H]
\includegraphics[width=1.0\textwidth]{pics/valentine.png}
\caption{The valentine simulation SDF defined in CoreCalc}
\label{valentine in corecalc}
\end{figure}


This function was benchmarked from 500 floats to 10000 floats. The intersection point between the GPU and the CPU is around 5000 floats.

\begin{figure}[H]
\includegraphics[width=1.0\textwidth]{pics/valentine.pdf}
\caption{The results of the valentine simulation}
\label{valentine chart in corecalc}
\end{figure}

\subsubsubsection{Approximation of $\pi$}
A Monte Carlo simulation can be used to approximate the value of $\pi$. This is done by generating a number of points within the square from (0, 0) to (1, 1), and afterwords counting the number of points that are within the inscribed circle of this square. The ratio between the counted points and the generated points should be $\pi/4$

We created a SDF $=IF(RAND()^2+RAND()^2<=1, 1, 0)$. Running this $n$ times and dividing the sum of the result with $n$ should approximate $\pi$. 

\begin{figure}[H]
\includegraphics[width=1.0\textwidth]{pics/pi.pdf}
\caption{The perfomance results of the $\pi$ approximation simulation}
\label{valentine chart in corecalc}
\end{figure}

This was benchmarked from 2000 to 100.000. The intersection was found around 50.000. 

\subsubsubsection{Commute time}
\begin{figure}[H]
\includegraphics[width=1.0\textwidth]{pics/commute.png}
\caption{The commute time simulation SDF defined in CoreCalc}
\label{commute time in corecalc}
\end{figure}
This testcase uses Monte Carlo simulations to predict the commute time to work. As seen in the above image, we have two road segments and a traffic light. At the first road segment we have a $10\%$ chance of hitting a traffic jam and at the traffic lights there is 120 seconds with red light and 30 seconds with green light. The second road sample chooses the time based on the probability in column J. In Excel this can be represented as a \keyword{Choose} formula or a \keyword{Lookup} formula, but as our experimental implementations does not support this, we simply use nested conditions. The simulation uses no input parameters.

\begin{figure}[H]
\includegraphics[width=1.0\textwidth]{pics/commute.pdf}
\caption{The performance results of commute example}
\label{commute chart in corecalc}
\end{figure}


\newpage
\section{Extra tables and charts}

\subsection{Test results from comparing different methods of representing constants in Accelerator}

\subsection{Test results from ...}